2026-04-10 19:20:35,491 [ 205122 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2026-04-10 19:20:35,491 [ 205122 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:79, check_args_and_update_paths) 2026-04-10 19:20:35,491 [ 205122 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:90, check_args_and_update_paths) 2026-04-10 19:20:35,491 [ 205122 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:92, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_8t0hyd --privileged --dns-search='.' --memory=30709035008 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2]' test_executable_user_defined_functions_config_reload/test.py::test test_explain_estimates/test.py::test_explain_estimates test_external_cluster/test.py::test_ddl test_external_cluster/test.py::test_ddl_replicated test_external_http_authenticator/test.py::test_basic_auth_failed test_external_http_authenticator/test.py::test_header_failed test_external_http_authenticator/test.py::test_session_settings_from_auth_response test_external_http_authenticator/test.py::test_user_create_basic_auth_pass test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 'test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node]' 'test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path]' test_filesystem_cache/test.py::test_custom_cached_disk test_filesystem_cache/test.py::test_dynamic_resize test_filesystem_cache/test.py::test_filesystem_cache_log test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges test_filesystem_cache/test.py::test_keep_up_size_ratio 'test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node]' test_filesystem_cache/test.py::test_system_sync_filesystem_cache test_grpc_protocol_ssl/test.py::test_insecure_channel test_grpc_protocol_ssl/test.py::test_secure_channel test_grpc_protocol_ssl/test.py::test_wrong_client_certificate test_hot_reload_storage_policy/test.py::test_hot_reload_policy test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table test_http_and_readonly/test.py::test_http_get_is_readonly test_http_native/test.py::test_http_native_returns_timezone test_inherit_multiple_profiles/test.py::test_combined_profile test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard test_insert_into_distributed_sync_async/test.py::test_insertion_sync test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout test_jbod_ha/test.py::test_jbod_ha test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin test_keeper_block_acl/test.py::test_keeper_block_acl test_keeper_compression/test_with_compression.py::test_select test_keeper_disks/test.py::test_logs_with_disks test_keeper_disks/test.py::test_snapshots_with_disks test_keeper_four_word_command/test_allow_list.py::test_allow_list test_keeper_internal_secure/test.py::test_secure_raft_works test_keeper_internal_secure/test.py::test_secure_raft_works_with_password test_keeper_memory_soft_limit/test.py::test_soft_limit_create test_keeper_mntr_pressure/test.py::test_aggressive_mntr test_keeper_nodes_move/test.py::test_node_move test_keeper_persistent_log/test.py::test_ephemeral_after_restart test_keeper_persistent_log/test.py::test_state_after_restart test_keeper_persistent_log/test.py::test_state_duplicate_restart test_keeper_session/test.py::test_session_close_shutdown test_keeper_session/test.py::test_session_timeout 'test_keeper_zookeeper_converter/test.py::test_acls[False]' 'test_keeper_zookeeper_converter/test.py::test_acls[True]' 'test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False]' 'test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True]' 'test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False]' 'test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True]' 'test_keeper_zookeeper_converter/test.py::test_smoke[False]' 'test_keeper_zookeeper_converter/test.py::test_smoke[True]' test_mask_sensitive_info/test.py::test_backup_table test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage test_mask_sensitive_info/test.py::test_backup_to_s3 test_mask_sensitive_info/test.py::test_create_alter_user test_mask_sensitive_info/test.py::test_create_database test_mask_sensitive_info/test.py::test_create_dictionary test_mask_sensitive_info/test.py::test_create_table test_mask_sensitive_info/test.py::test_encryption_functions test_mask_sensitive_info/test.py::test_iceberg_cluster_function test_mask_sensitive_info/test.py::test_on_cluster test_mask_sensitive_info/test.py::test_table_function_ways_to_call test_mask_sensitive_info/test.py::test_table_functions test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers test_reload_clusters_config/test.py::test_add_cluster test_reload_clusters_config/test.py::test_delete_cluster test_reload_clusters_config/test.py::test_simple_reload test_reload_clusters_config/test.py::test_update_one_cluster 'test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain]' test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] test_mask_sensitive_info/test.py::test_backup_table test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard test_external_http_authenticator/test.py::test_basic_auth_failed test_keeper_zookeeper_converter/test.py::test_acls[False] test_reload_clusters_config/test.py::test_add_cluster [gw6] [ 1%] PASSED test_external_http_authenticator/test.py::test_basic_auth_failed test_external_http_authenticator/test.py::test_header_failed [gw9] [ 2%] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes [gw6] [ 3%] PASSED test_external_http_authenticator/test.py::test_header_failed test_external_http_authenticator/test.py::test_session_settings_from_auth_response [gw9] [ 4%] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk [gw7] [ 5%] PASSED test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard test_insert_into_distributed_sync_async/test.py::test_insertion_sync [gw1] [ 6%] PASSED test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] [gw3] [ 7%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] [gw9] [ 8%] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin [gw6] [ 9%] PASSED test_external_http_authenticator/test.py::test_session_settings_from_auth_response test_external_http_authenticator/test.py::test_user_create_basic_auth_pass [gw3] [ 10%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] [gw3] [ 11%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] [gw3] [ 12%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] [gw9] [ 13%] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin [gw3] [ 14%] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] test_external_cluster/test.py::test_ddl [gw6] [ 15%] PASSED test_external_http_authenticator/test.py::test_user_create_basic_auth_pass test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass [gw7] [ 16%] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout [gw6] [ 17%] PASSED test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass [gw1] [ 18%] PASSED test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] test_filesystem_cache/test.py::test_custom_cached_disk [gw7] [ 19%] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout [gw7] [ 20%] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout test_grpc_protocol_ssl/test.py::test_insecure_channel test_keeper_persistent_log/test.py::test_ephemeral_after_restart [gw7] [ 21%] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout test_keeper_disks/test.py::test_logs_with_disks [gw2] [ 22%] PASSED test_keeper_zookeeper_converter/test.py::test_acls[False] test_keeper_zookeeper_converter/test.py::test_acls[True] [gw4] [ 23%] PASSED test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] [gw4] [ 24%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] [gw4] [ 25%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection [gw9] [ 26%] PASSED test_grpc_protocol_ssl/test.py::test_insecure_channel test_grpc_protocol_ssl/test.py::test_secure_channel [gw9] [ 27%] PASSED test_grpc_protocol_ssl/test.py::test_secure_channel test_grpc_protocol_ssl/test.py::test_wrong_client_certificate [gw9] [ 28%] PASSED test_grpc_protocol_ssl/test.py::test_wrong_client_certificate [gw6] [ 29%] PASSED test_keeper_persistent_log/test.py::test_ephemeral_after_restart test_keeper_persistent_log/test.py::test_state_after_restart [gw3] [ 30%] PASSED test_external_cluster/test.py::test_ddl test_external_cluster/test.py::test_ddl_replicated [gw3] [ 31%] PASSED test_external_cluster/test.py::test_ddl_replicated test_keeper_session/test.py::test_session_close_shutdown [gw6] [ 32%] PASSED test_keeper_persistent_log/test.py::test_state_after_restart test_keeper_persistent_log/test.py::test_state_duplicate_restart [gw5] [ 33%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] [gw5] [ 34%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format test_hot_reload_storage_policy/test.py::test_hot_reload_policy [gw5] [ 35%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] [gw5] [ 36%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] [gw1] [ 37%] PASSED test_filesystem_cache/test.py::test_custom_cached_disk [gw2] [ 38%] PASSED test_keeper_zookeeper_converter/test.py::test_acls[True] test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] [gw5] [ 39%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] test_filesystem_cache/test.py::test_dynamic_resize [gw5] [ 40%] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] [gw1] [ 41%] PASSED test_filesystem_cache/test.py::test_dynamic_resize test_filesystem_cache/test.py::test_filesystem_cache_log [gw4] [ 42%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] [gw1] [ 43%] PASSED test_filesystem_cache/test.py::test_filesystem_cache_log test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges [gw6] [ 44%] PASSED test_keeper_persistent_log/test.py::test_state_duplicate_restart [gw4] [ 45%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task [gw3] [ 46%] PASSED test_hot_reload_storage_policy/test.py::test_hot_reload_policy test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table [gw4] [ 47%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] test_quorum_inserts/test.py::test_insert_quorum_with_ttl [gw7] [ 48%] PASSED test_keeper_disks/test.py::test_logs_with_disks test_keeper_disks/test.py::test_snapshots_with_disks [gw2] [ 49%] PASSED test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] [gw3] [ 50%] PASSED test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table test_explain_estimates/test.py::test_explain_estimates [gw9] [ 51%] PASSED test_keeper_session/test.py::test_session_close_shutdown test_keeper_session/test.py::test_session_timeout [gw9] [ 52%] PASSED test_keeper_session/test.py::test_session_timeout test_executable_user_defined_functions_config_reload/test.py::test test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts [gw4] [ 53%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica [gw1] [ 54%] PASSED test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges test_filesystem_cache/test.py::test_keep_up_size_ratio [gw4] [ 55%] PASSED test_quorum_inserts/test.py::test_simple_add_replica [gw0] [ 56%] PASSED test_mask_sensitive_info/test.py::test_backup_table test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage [gw7] [ 57%] PASSED test_keeper_disks/test.py::test_snapshots_with_disks [gw0] [ 58%] PASSED test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage test_mask_sensitive_info/test.py::test_backup_to_s3 [gw3] [ 59%] PASSED test_explain_estimates/test.py::test_explain_estimates [gw1] [ 60%] PASSED test_filesystem_cache/test.py::test_keep_up_size_ratio test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] [gw6] [ 61%] PASSED test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table test_inherit_multiple_profiles/test.py::test_combined_profile [gw2] [ 62%] PASSED test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] [gw5] [ 63%] PASSED test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts [gw6] [ 64%] PASSED test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table [gw1] [ 65%] PASSED test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] test_filesystem_cache/test.py::test_system_sync_filesystem_cache test_http_native/test.py::test_http_native_returns_timezone [gw3] [ 66%] PASSED test_inherit_multiple_profiles/test.py::test_combined_profile test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success [gw1] [ 67%] PASSED test_filesystem_cache/test.py::test_system_sync_filesystem_cache [gw2] [ 68%] PASSED test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] test_jbod_ha/test.py::test_jbod_ha test_keeper_internal_secure/test.py::test_secure_raft_works test_http_and_readonly/test.py::test_http_get_is_readonly test_keeper_mntr_pressure/test.py::test_aggressive_mntr [gw9] [ 69%] PASSED test_executable_user_defined_functions_config_reload/test.py::test test_keeper_block_acl/test.py::test_keeper_block_acl [gw5] [ 70%] PASSED test_http_and_readonly/test.py::test_http_get_is_readonly [gw6] [ 71%] PASSED test_http_native/test.py::test_http_native_returns_timezone test_keeper_four_word_command/test_allow_list.py::test_allow_list [gw4] [ 72%] PASSED test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success test_keeper_nodes_move/test.py::test_node_move test_keeper_memory_soft_limit/test.py::test_soft_limit_create [gw2] [ 73%] PASSED test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] test_keeper_zookeeper_converter/test.py::test_smoke[False] [gw5] [ 74%] PASSED test_keeper_four_word_command/test_allow_list.py::test_allow_list [gw7] [ 75%] PASSED test_keeper_internal_secure/test.py::test_secure_raft_works test_keeper_internal_secure/test.py::test_secure_raft_works_with_password [gw2] [ 76%] PASSED test_keeper_zookeeper_converter/test.py::test_smoke[False] test_keeper_zookeeper_converter/test.py::test_smoke[True] [gw0] [ 77%] PASSED test_mask_sensitive_info/test.py::test_backup_to_s3 test_mask_sensitive_info/test.py::test_create_alter_user [gw0] [ 78%] PASSED test_mask_sensitive_info/test.py::test_create_alter_user test_mask_sensitive_info/test.py::test_create_database [gw0] [ 79%] PASSED test_mask_sensitive_info/test.py::test_create_database test_mask_sensitive_info/test.py::test_create_dictionary [gw3] [ 80%] PASSED test_jbod_ha/test.py::test_jbod_ha [gw0] [ 81%] PASSED test_mask_sensitive_info/test.py::test_create_dictionary test_mask_sensitive_info/test.py::test_create_table test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback [gw0] [ 82%] FAILED test_mask_sensitive_info/test.py::test_create_table test_mask_sensitive_info/test.py::test_encryption_functions [gw0] [ 83%] PASSED test_mask_sensitive_info/test.py::test_encryption_functions test_mask_sensitive_info/test.py::test_iceberg_cluster_function [gw2] [ 84%] PASSED test_keeper_zookeeper_converter/test.py::test_smoke[True] [gw6] [ 85%] PASSED test_keeper_nodes_move/test.py::test_node_move [gw9] [ 86%] PASSED test_keeper_block_acl/test.py::test_keeper_block_acl test_keeper_compression/test_with_compression.py::test_select [gw0] [ 87%] PASSED test_mask_sensitive_info/test.py::test_iceberg_cluster_function test_mask_sensitive_info/test.py::test_on_cluster [gw3] [ 88%] PASSED test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback [gw0] [ 89%] PASSED test_mask_sensitive_info/test.py::test_on_cluster test_mask_sensitive_info/test.py::test_table_function_ways_to_call [gw9] [ 90%] PASSED test_keeper_compression/test_with_compression.py::test_select [gw8] [ 91%] PASSED test_reload_clusters_config/test.py::test_add_cluster test_reload_clusters_config/test.py::test_delete_cluster [gw0] [ 92%] PASSED test_mask_sensitive_info/test.py::test_table_function_ways_to_call test_mask_sensitive_info/test.py::test_table_functions [gw0] [ 93%] FAILED test_mask_sensitive_info/test.py::test_table_functions [gw7] [ 94%] PASSED test_keeper_internal_secure/test.py::test_secure_raft_works_with_password [gw1] [ 95%] PASSED test_keeper_mntr_pressure/test.py::test_aggressive_mntr test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers [gw1] [ 96%] PASSED test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers [gw8] [ 97%] FAILED test_reload_clusters_config/test.py::test_delete_cluster test_reload_clusters_config/test.py::test_simple_reload [gw8] [ 98%] PASSED test_reload_clusters_config/test.py::test_simple_reload test_reload_clusters_config/test.py::test_update_one_cluster [gw4] [ 99%] PASSED test_keeper_memory_soft_limit/test.py::test_soft_limit_create [gw8] [100%] FAILED test_reload_clusters_config/test.py::test_update_one_cluster =================================== FAILURES =================================== ______________________________ test_create_table _______________________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 def test_create_table(): password = new_password() azure_conn_string = cluster.env_variables["AZURITE_CONNECTION_STRING"] account_key_pattern = re.compile("AccountKey=.*?(;|$)") masked_azure_conn_string = re.sub( account_key_pattern, "AccountKey=[HIDDEN]\\1", azure_conn_string ) masked_sas_conn_string = 'BlobEndpoint=https://my-endpoint/;SharedAccessSignature=[HIDDEN]' azure_storage_account_url = cluster.env_variables["AZURITE_STORAGE_ACCOUNT_URL"] azure_account_name = "devstoreaccount1" azure_account_key = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==" table_engines = [ f"MySQL('mysql80:3306', 'mysql_db', 'mysql_table', 'mysql_user', '{password}')", f"PostgreSQL('postgres1:5432', 'postgres_db', 'postgres_table', 'postgres_user', '{password}')", f"MongoDB('mongo1:27017', 'mongo_db', 'mongo_col', 'mongo_user', '{password}')", f"S3('http://minio1:9001/root/data/test1.csv')", f"S3('http://minio1:9001/root/data/test2.csv', 'CSV')", f"S3('http://minio1:9001/root/data/test3.csv.gz', 'CSV', 'gzip')", f"S3('http://minio1:9001/root/data/test4.csv', 'minio', '{password}', 'CSV')", f"S3('http://minio1:9001/root/data/test5.csv.gz', 'minio', '{password}', 'CSV', 'gzip')", f"MySQL(named_collection_1, host = 'mysql80', port = 3306, database = 'mysql_db', table = 'mysql_table', user = 'mysql_user', password = '{password}')", f"MySQL(named_collection_2, database = 'mysql_db', host = 'mysql80', port = 3306, password = '{password}', table = 'mysql_table', user = 'mysql_user')", f"MySQL(named_collection_3, database = 'mysql_db', host = 'mysql80', port = 3306, table = 'mysql_table')", f"PostgreSQL(named_collection_4, host = 'postgres1', port = 5432, database = 'postgres_db', table = 'postgres_table', user = 'postgres_user', password = '{password}')", f"MongoDB(named_collection_5, host = 'mongo1', port = 5432, database = 'mongo_db', collection = 'mongo_col', user = 'mongo_user', password = '{password}')", f"S3(named_collection_6, url = 'http://minio1:9001/root/data/test8.csv', access_key_id = 'minio', secret_access_key = '{password}', format = 'CSV')", f"S3('http://minio1:9001/root/data/test9.csv.gz', 'NOSIGN', 'CSV', 'gzip')", f"S3('http://minio1:9001/root/data/test10.csv.gz', 'minio', '{password}')", ( f"DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", "DNS_ERROR", ), f"S3Queue('http://minio1:9001/root/data/', 'CSV') settings mode = 'ordered'", f"S3Queue('http://minio1:9001/root/data/', 'CSV', 'gzip') settings mode = 'ordered'", f"S3Queue('http://minio1:9001/root/data/', 'minio', '{password}', 'CSV') settings mode = 'ordered'", f"S3Queue('http://minio1:9001/root/data/', 'minio', '{password}', 'CSV', 'gzip') settings mode = 'ordered'", ( f"Iceberg('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", "DNS_ERROR", ), ( f"IcebergS3('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", "DNS_ERROR", ), f"AzureBlobStorage('{azure_conn_string}', 'cont', 'test_simple.csv', 'CSV')", f"AzureBlobStorage('{azure_conn_string}', 'cont', 'test_simple_1.csv', 'CSV', 'none')", f"AzureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_2.csv', '{azure_account_name}', '{azure_account_key}')", f"AzureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_3.csv', '{azure_account_name}', '{azure_account_key}', 'CSV')", f"AzureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_4.csv', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none')", f"AzureQueue('{azure_conn_string}', 'cont', '*', 'CSV') SETTINGS mode = 'unordered'", f"AzureQueue('{azure_conn_string}', 'cont', '*', 'CSV', 'none') SETTINGS mode = 'unordered'", f"AzureQueue('{azure_storage_account_url}', 'cont', '*', '{azure_account_name}', '{azure_account_key}', 'CSV') SETTINGS mode = 'unordered'", f"AzureQueue('{azure_storage_account_url}', 'cont', '*', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none') SETTINGS mode = 'unordered'", ( f"AzureBlobStorage('BlobEndpoint=https://my-endpoint/;SharedAccessSignature=sp=r&st=2025-09-29T14:58:11Z&se=2025-09-29T00:00:00Z&spr=https&sv=2022-11-02&sr=c&sig=SECRET%SECRET%SECRET%SECRET', 'exampledatasets', 'example.csv')", "STD_EXCEPTION", ), f"Kafka() SETTINGS kafka_broker_list = '127.0.0.1', kafka_topic_list = 'topic', kafka_group_name = 'group', kafka_format = 'JSONEachRow', format_avro_schema_registry_url = 'http://schema_user:{password}@'", f"Kafka() SETTINGS kafka_broker_list = '127.0.0.1', kafka_topic_list = 'topic', kafka_group_name = 'group', kafka_format = 'JSONEachRow', format_avro_schema_registry_url = 'http://schema_user:{password}@domain.com'", ] def make_test_case(i): table_name = f"table{i}" table_engine = table_engines[i] error = None if isinstance(table_engine, tuple): table_engine, error = table_engine query = f"CREATE TABLE {table_name} (x int) ENGINE = {table_engine}" return table_name, query, error # Generate test cases as a list of tuples (table_name, query, error). test_cases = [make_test_case(i) for i in range(len(table_engines))] for table_name, query, error in test_cases: if error: > assert error in node.query_and_get_error(query) E assert 'DNS_ERROR' in "Received exception from server (version 25.3.8):\nCode: 56. DB::Exception: Received from 172.16.2.6:9000. DB::Exception: Unknown table engine DeltaLake. Stack trace:\n\n0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000d7ed77b\n1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008292ccc\n2. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x00000000082a8d6b\n3. DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel, bool) const @ 0x00000000122c85cf\n4. DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x000000001187aa00\n5. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000001186f6c2\n6. DB::InterpreterCreateQuery::execute() @ 0x0000000011882c18\n7. DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x0000000011cadfab\n8. DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000011ca89c4\n9. DB::TCPHandler::runImpl() @ 0x0000000012fed5ac\n10. DB::TCPHandler::run() @ 0x000000001300b799\n11. Poco::Net::TCPServerConnection::start() @ 0x0000000016744027\n12. Poco::Net::TCPServerDispatcher::run() @ 0x0000000016744479\n13. Poco::PooledThread::run() @ 0x00000000167107db\n14. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001670ecbd\n15. ? @ 0x00007f2188442ac3\n16. ? @ 0x00007f21884d4850\n. (UNKNOWN_STORAGE)\n(query: CREATE TABLE table16 (x int) ENGINE = DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '3Q75MM4W5SHN2CF1'))\n" E + where "Received exception from server (version 25.3.8):\nCode: 56. DB::Exception: Received from 172.16.2.6:9000. DB::Exception: Unknown table engine DeltaLake. Stack trace:\n\n0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000d7ed77b\n1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008292ccc\n2. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&) @ 0x00000000082a8d6b\n3. DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel, bool) const @ 0x00000000122c85cf\n4. DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x000000001187aa00\n5. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000001186f6c2\n6. DB::InterpreterCreateQuery::execute() @ 0x0000000011882c18\n7. DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x0000000011cadfab\n8. DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000011ca89c4\n9. DB::TCPHandler::runImpl() @ 0x0000000012fed5ac\n10. DB::TCPHandler::run() @ 0x000000001300b799\n11. Poco::Net::TCPServerConnection::start() @ 0x0000000016744027\n12. Poco::Net::TCPServerDispatcher::run() @ 0x0000000016744479\n13. Poco::PooledThread::run() @ 0x00000000167107db\n14. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001670ecbd\n15. ? @ 0x00007f2188442ac3\n16. ? @ 0x00007f21884d4850\n. (UNKNOWN_STORAGE)\n(query: CREATE TABLE table16 (x int) ENGINE = DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '3Q75MM4W5SHN2CF1'))\n" = >("CREATE TABLE table16 (x int) ENGINE = DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '3Q75MM4W5SHN2CF1')") E + where > = .query_and_get_error test_mask_sensitive_info/test.py:309: AssertionError ------------------------------ Captured log call ------------------------------- 2026-04-10 19:22:53.637000 [ 639 ] DEBUG : Executing query CREATE TABLE table0 (x int) ENGINE = MySQL('mysql80:3306', 'mysql_db', 'mysql_table', 'mysql_user', '3Q75MM4W5SHN2CF1') on node (cluster.py:3658, query) 2026-04-10 19:22:53.702000 [ 639 ] DEBUG : Executing query CREATE TABLE table1 (x int) ENGINE = PostgreSQL('postgres1:5432', 'postgres_db', 'postgres_table', 'postgres_user', '3Q75MM4W5SHN2CF1') on node (cluster.py:3658, query) 2026-04-10 19:22:53.818000 [ 639 ] DEBUG : Executing query CREATE TABLE table2 (x int) ENGINE = MongoDB('mongo1:27017', 'mongo_db', 'mongo_col', 'mongo_user', '3Q75MM4W5SHN2CF1') on node (cluster.py:3658, query) 2026-04-10 19:22:53.883000 [ 639 ] DEBUG : Executing query CREATE TABLE table3 (x int) ENGINE = S3('http://minio1:9001/root/data/test1.csv') on node (cluster.py:3658, query) 2026-04-10 19:22:53.998000 [ 639 ] DEBUG : Executing query CREATE TABLE table4 (x int) ENGINE = S3('http://minio1:9001/root/data/test2.csv', 'CSV') on node (cluster.py:3658, query) 2026-04-10 19:22:54.218000 [ 639 ] DEBUG : Executing query CREATE TABLE table5 (x int) ENGINE = S3('http://minio1:9001/root/data/test3.csv.gz', 'CSV', 'gzip') on node (cluster.py:3658, query) 2026-04-10 19:22:54.334000 [ 639 ] DEBUG : Executing query CREATE TABLE table6 (x int) ENGINE = S3('http://minio1:9001/root/data/test4.csv', 'minio', '3Q75MM4W5SHN2CF1', 'CSV') on node (cluster.py:3658, query) 2026-04-10 19:22:54.451000 [ 639 ] DEBUG : Executing query CREATE TABLE table7 (x int) ENGINE = S3('http://minio1:9001/root/data/test5.csv.gz', 'minio', '3Q75MM4W5SHN2CF1', 'CSV', 'gzip') on node (cluster.py:3658, query) 2026-04-10 19:22:54.617000 [ 639 ] DEBUG : Executing query CREATE TABLE table8 (x int) ENGINE = MySQL(named_collection_1, host = 'mysql80', port = 3306, database = 'mysql_db', table = 'mysql_table', user = 'mysql_user', password = '3Q75MM4W5SHN2CF1') on node (cluster.py:3658, query) 2026-04-10 19:22:54.683000 [ 639 ] DEBUG : Executing query CREATE TABLE table9 (x int) ENGINE = MySQL(named_collection_2, database = 'mysql_db', host = 'mysql80', port = 3306, password = '3Q75MM4W5SHN2CF1', table = 'mysql_table', user = 'mysql_user') on node (cluster.py:3658, query) 2026-04-10 19:22:54.748000 [ 639 ] DEBUG : Executing query CREATE TABLE table10 (x int) ENGINE = MySQL(named_collection_3, database = 'mysql_db', host = 'mysql80', port = 3306, table = 'mysql_table') on node (cluster.py:3658, query) 2026-04-10 19:22:54.864000 [ 639 ] DEBUG : Executing query CREATE TABLE table11 (x int) ENGINE = PostgreSQL(named_collection_4, host = 'postgres1', port = 5432, database = 'postgres_db', table = 'postgres_table', user = 'postgres_user', password = '3Q75MM4W5SHN2CF1') on node (cluster.py:3658, query) 2026-04-10 19:22:54.931000 [ 639 ] DEBUG : Executing query CREATE TABLE table12 (x int) ENGINE = MongoDB(named_collection_5, host = 'mongo1', port = 5432, database = 'mongo_db', collection = 'mongo_col', user = 'mongo_user', password = '3Q75MM4W5SHN2CF1') on node (cluster.py:3658, query) 2026-04-10 19:22:54.997000 [ 639 ] DEBUG : Executing query CREATE TABLE table13 (x int) ENGINE = S3(named_collection_6, url = 'http://minio1:9001/root/data/test8.csv', access_key_id = 'minio', secret_access_key = '3Q75MM4W5SHN2CF1', format = 'CSV') on node (cluster.py:3658, query) 2026-04-10 19:22:55.062000 [ 639 ] DEBUG : Executing query CREATE TABLE table14 (x int) ENGINE = S3('http://minio1:9001/root/data/test9.csv.gz', 'NOSIGN', 'CSV', 'gzip') on node (cluster.py:3658, query) 2026-04-10 19:22:55.128000 [ 639 ] DEBUG : Executing query CREATE TABLE table15 (x int) ENGINE = S3('http://minio1:9001/root/data/test10.csv.gz', 'minio', '3Q75MM4W5SHN2CF1') on node (cluster.py:3658, query) 2026-04-10 19:22:55.193000 [ 639 ] DEBUG : Executing query CREATE TABLE table16 (x int) ENGINE = DeltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '3Q75MM4W5SHN2CF1') on node (cluster.py:3741, query_and_get_error) _____________________________ test_table_functions _____________________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 def test_table_functions(): password = new_password() azure_conn_string = cluster.env_variables["AZURITE_CONNECTION_STRING"] account_key_pattern = re.compile("AccountKey=.*?(;|$)") masked_azure_conn_string = re.sub( account_key_pattern, "AccountKey=[HIDDEN]\\1", azure_conn_string ) azure_storage_account_url = cluster.env_variables["AZURITE_STORAGE_ACCOUNT_URL"] azure_account_name = "devstoreaccount1" azure_account_key = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==" table_functions = [ f"mysql('mysql80:3306', 'mysql_db', 'mysql_table', 'mysql_user', '{password}')", f"postgresql('postgres1:5432', 'postgres_db', 'postgres_table', 'postgres_user', '{password}')", f"mongodb('mongo1:27017', 'mongo_db', 'mongo_col', 'mongo_user', '{password}', 'x int')", f"s3('http://minio1:9001/root/data/test1.csv')", f"s3('http://minio1:9001/root/data/test2.csv', 'CSV')", f"s3('http://minio1:9001/root/data/test3.csv', 'minio', '{password}')", f"s3('http://minio1:9001/root/data/test4.csv', 'CSV', 'x int')", f"s3('http://minio1:9001/root/data/test5.csv.gz', 'CSV', 'x int', 'gzip')", f"s3('http://minio1:9001/root/data/test6.csv', 'minio', '{password}', 'CSV')", f"s3('http://minio1:9001/root/data/test7.csv', 'minio', '{password}', 'CSV', 'x int')", f"s3('http://minio1:9001/root/data/test8.csv.gz', 'minio', '{password}', 'CSV', 'x int', 'gzip')", f"remote('127.{{2..11}}', default.remote_table)", f"remote('127.{{2..11}}', default.remote_table, rand())", f"remote('127.{{2..11}}', default.remote_table, 'remote_user')", f"remote('127.{{2..11}}', default.remote_table, 'remote_user', '{password}')", f"remote('127.{{2..11}}', default.remote_table, 'remote_user', rand())", f"remote('127.{{2..11}}', default.remote_table, 'remote_user', '{password}', rand())", f"remote('127.{{2..11}}', 'default.remote_table', 'remote_user', '{password}', rand())", f"remote('127.{{2..11}}', 'default', 'remote_table', 'remote_user', '{password}', rand())", f"remote('127.{{2..11}}', numbers(10), 'remote_user', '{password}', rand())", f"remoteSecure('127.{{2..11}}', 'default', 'remote_table', 'remote_user', '{password}')", f"remoteSecure('127.{{2..11}}', 'default', 'remote_table', 'remote_user', rand())", f"mysql(named_collection_1, host = 'mysql80', port = 3306, database = 'mysql_db', table = 'mysql_table', user = 'mysql_user', password = '{password}')", f"postgresql(named_collection_2, password = '{password}', host = 'postgres1', port = 5432, database = 'postgres_db', table = 'postgres_table', user = 'postgres_user')", f"s3(named_collection_2, url = 'http://minio1:9001/root/data/test4.csv', access_key_id = 'minio', secret_access_key = '{password}')", f"remote(named_collection_6, addresses_expr = '127.{{2..11}}', database = 'default', table = 'remote_table', user = 'remote_user', password = '{password}', sharding_key = rand())", f"remoteSecure(named_collection_6, addresses_expr = '127.{{2..11}}', database = 'default', table = 'remote_table', user = 'remote_user', password = '{password}')", f"s3('http://minio1:9001/root/data/test9.csv.gz', 'NOSIGN', 'CSV')", f"s3('http://minio1:9001/root/data/test10.csv.gz', 'minio', '{password}')", f"deltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", f"azureBlobStorage('{azure_conn_string}', 'cont', 'test_simple.csv', 'CSV')", f"azureBlobStorage('{azure_conn_string}', 'cont', 'test_simple_1.csv', 'CSV', 'none')", f"azureBlobStorage('{azure_conn_string}', 'cont', 'test_simple_2.csv', 'CSV', 'none', 'auto')", f"azureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_3.csv', '{azure_account_name}', '{azure_account_key}')", f"azureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_4.csv', '{azure_account_name}', '{azure_account_key}', 'CSV')", f"azureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_5.csv', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none')", f"azureBlobStorage('{azure_storage_account_url}', 'cont', 'test_simple_6.csv', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none', 'auto')", f"azureBlobStorage(named_collection_2, connection_string = '{azure_conn_string}', container = 'cont', blob_path = 'test_simple_7.csv', format = 'CSV')", f"azureBlobStorage(named_collection_2, storage_account_url = '{azure_storage_account_url}', container = 'cont', blob_path = 'test_simple_8.csv', account_name = '{azure_account_name}', account_key = '{azure_account_key}')", f"iceberg('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", f"gcs('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", f"icebergS3('http://minio1:9001/root/data/test11.csv.gz', 'minio', '{password}')", f"icebergAzure('{azure_storage_account_url}', 'cont', 'test_simple_6.csv', '{azure_account_name}', '{azure_account_key}', 'CSV', 'none', 'auto')", ] def make_test_case(i): table_name = f"tablefunc{i}" table_function = table_functions[i] error = None if isinstance(table_function, tuple): table_function, error = table_function query = f"CREATE TABLE {table_name} (x int) AS {table_function}" return table_name, query, error # Generate test cases as a list of tuples (table_name, query, error). test_cases = [make_test_case(i) for i in range(len(table_functions))] for table_name, query, error in test_cases: > node.query(query) test_mask_sensitive_info/test.py:503: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3659: in query return self.client.query( helpers/client.py:39: in wrap return func(self, *args, **kwargs) helpers/client.py:79: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 46, stderr: Received exception from server (version 25.3.8): E Code: 46. DB::Exception: Received from 172.16.2.6:9000. DB::Exception: Unknown table function deltaLake. Stack trace: E E 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000d7ed77b E 1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000008292ccc E 2. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String const&) @ 0x0000000008b8f68b E 3. DB::TableFunctionFactory::get(std::shared_ptr const&, std::shared_ptr) const @ 0x0000000010068628 E 4. DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr>&, DB::LoadingStrictnessLevel) @ 0x000000001187a81b E 5. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x000000001186f6c2 E 6. DB::InterpreterCreateQuery::execute() @ 0x0000000011882c18 E 7. DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*, std::shared_ptr&) @ 0x0000000011cadfab E 8. DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000011ca89c4 E 9. DB::TCPHandler::runImpl() @ 0x0000000012fed5ac E 10. DB::TCPHandler::run() @ 0x000000001300b799 E 11. Poco::Net::TCPServerConnection::start() @ 0x0000000016744027 E 12. Poco::Net::TCPServerDispatcher::run() @ 0x0000000016744479 E 13. Poco::PooledThread::run() @ 0x00000000167107db E 14. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001670ecbd E 15. ? @ 0x00007f2188442ac3 E 16. ? @ 0x00007f21884d4850 E . (UNKNOWN_FUNCTION) E (query: CREATE TABLE tablefunc29 (x int) AS deltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', 'IU1HF6VB92HZ0JDL')) helpers/client.py:248: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2026-04-10 19:23:31.710000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc0 (x int) AS mysql('mysql80:3306', 'mysql_db', 'mysql_table', 'mysql_user', 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:31.826000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc1 (x int) AS postgresql('postgres1:5432', 'postgres_db', 'postgres_table', 'postgres_user', 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:31.891000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc2 (x int) AS mongodb('mongo1:27017', 'mongo_db', 'mongo_col', 'mongo_user', 'IU1HF6VB92HZ0JDL', 'x int') on node (cluster.py:3658, query) 2026-04-10 19:23:32.067000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc3 (x int) AS s3('http://minio1:9001/root/data/test1.csv') on node (cluster.py:3658, query) 2026-04-10 19:23:32.132000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc4 (x int) AS s3('http://minio1:9001/root/data/test2.csv', 'CSV') on node (cluster.py:3658, query) 2026-04-10 19:23:32.247000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc5 (x int) AS s3('http://minio1:9001/root/data/test3.csv', 'minio', 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:32.363000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc6 (x int) AS s3('http://minio1:9001/root/data/test4.csv', 'CSV', 'x int') on node (cluster.py:3658, query) 2026-04-10 19:23:32.478000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc7 (x int) AS s3('http://minio1:9001/root/data/test5.csv.gz', 'CSV', 'x int', 'gzip') on node (cluster.py:3658, query) 2026-04-10 19:23:32.594000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc8 (x int) AS s3('http://minio1:9001/root/data/test6.csv', 'minio', 'IU1HF6VB92HZ0JDL', 'CSV') on node (cluster.py:3658, query) 2026-04-10 19:23:32.711000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc9 (x int) AS s3('http://minio1:9001/root/data/test7.csv', 'minio', 'IU1HF6VB92HZ0JDL', 'CSV', 'x int') on node (cluster.py:3658, query) 2026-04-10 19:23:32.776000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc10 (x int) AS s3('http://minio1:9001/root/data/test8.csv.gz', 'minio', 'IU1HF6VB92HZ0JDL', 'CSV', 'x int', 'gzip') on node (cluster.py:3658, query) 2026-04-10 19:23:32.841000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc11 (x int) AS remote('127.{2..11}', default.remote_table) on node (cluster.py:3658, query) 2026-04-10 19:23:32.906000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc12 (x int) AS remote('127.{2..11}', default.remote_table, rand()) on node (cluster.py:3658, query) 2026-04-10 19:23:33.021000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc13 (x int) AS remote('127.{2..11}', default.remote_table, 'remote_user') on node (cluster.py:3658, query) 2026-04-10 19:23:33.138000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc14 (x int) AS remote('127.{2..11}', default.remote_table, 'remote_user', 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:33.203000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc15 (x int) AS remote('127.{2..11}', default.remote_table, 'remote_user', rand()) on node (cluster.py:3658, query) 2026-04-10 19:23:33.268000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc16 (x int) AS remote('127.{2..11}', default.remote_table, 'remote_user', 'IU1HF6VB92HZ0JDL', rand()) on node (cluster.py:3658, query) 2026-04-10 19:23:33.383000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc17 (x int) AS remote('127.{2..11}', 'default.remote_table', 'remote_user', 'IU1HF6VB92HZ0JDL', rand()) on node (cluster.py:3658, query) 2026-04-10 19:23:33.448000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc18 (x int) AS remote('127.{2..11}', 'default', 'remote_table', 'remote_user', 'IU1HF6VB92HZ0JDL', rand()) on node (cluster.py:3658, query) 2026-04-10 19:23:33.515000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc19 (x int) AS remote('127.{2..11}', numbers(10), 'remote_user', 'IU1HF6VB92HZ0JDL', rand()) on node (cluster.py:3658, query) 2026-04-10 19:23:33.630000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc20 (x int) AS remoteSecure('127.{2..11}', 'default', 'remote_table', 'remote_user', 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:33.746000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc21 (x int) AS remoteSecure('127.{2..11}', 'default', 'remote_table', 'remote_user', rand()) on node (cluster.py:3658, query) 2026-04-10 19:23:33.862000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc22 (x int) AS mysql(named_collection_1, host = 'mysql80', port = 3306, database = 'mysql_db', table = 'mysql_table', user = 'mysql_user', password = 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:33.978000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc23 (x int) AS postgresql(named_collection_2, password = 'IU1HF6VB92HZ0JDL', host = 'postgres1', port = 5432, database = 'postgres_db', table = 'postgres_table', user = 'postgres_user') on node (cluster.py:3658, query) 2026-04-10 19:23:34.097000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc24 (x int) AS s3(named_collection_2, url = 'http://minio1:9001/root/data/test4.csv', access_key_id = 'minio', secret_access_key = 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:34.162000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc25 (x int) AS remote(named_collection_6, addresses_expr = '127.{2..11}', database = 'default', table = 'remote_table', user = 'remote_user', password = 'IU1HF6VB92HZ0JDL', sharding_key = rand()) on node (cluster.py:3658, query) 2026-04-10 19:23:34.278000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc26 (x int) AS remoteSecure(named_collection_6, addresses_expr = '127.{2..11}', database = 'default', table = 'remote_table', user = 'remote_user', password = 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:34.393000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc27 (x int) AS s3('http://minio1:9001/root/data/test9.csv.gz', 'NOSIGN', 'CSV') on node (cluster.py:3658, query) 2026-04-10 19:23:34.458000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc28 (x int) AS s3('http://minio1:9001/root/data/test10.csv.gz', 'minio', 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) 2026-04-10 19:23:34.524000 [ 639 ] DEBUG : Executing query CREATE TABLE tablefunc29 (x int) AS deltaLake('http://minio1:9001/root/data/test11.csv.gz', 'minio', 'IU1HF6VB92HZ0JDL') on node (cluster.py:3658, query) ---------------------------- Captured log teardown ----------------------------- 2026-04-10 19:23:34.756000 [ 639 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw0/.env --project-name roottestmasksensitiveinfo-gw0 --file /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw0/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 19:23:49.514000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:49.515000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:49.515000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:49.515000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:49.515000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:49.515000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-azurite1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:49.515000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:49.515000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:49.515000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:49.516000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-azurite1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:49.516000 [ 639 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw0/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw0/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 19:23:49.530000 [ 639 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw0/.env --project-name roottestmasksensitiveinfo-gw0 --file /ClickHouse/tests/integration/test_mask_sensitive_info/_instances-0-gw0/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 19:23:50.257000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:50.258000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:50.258000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-node-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:23:50.258000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-node-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:23:50.258000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:50.258000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:50.258000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:50.258000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-azurite1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:23:50.259000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:50.259000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:23:50.259000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:50.259000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:23:50.259000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:50.259000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:23:50.259000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-azurite1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:23:50.260000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-azurite1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:23:50.260000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:23:50.260000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:23:50.260000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-zoo2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:23:50.260000 [ 639 ] DEBUG : Stderr: Container roottestmasksensitiveinfo-gw0-azurite1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:23:50.260000 [ 639 ] DEBUG : Stderr: Volume roottestmasksensitiveinfo-gw0_data1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:23:50.260000 [ 639 ] DEBUG : Stderr: Network roottestmasksensitiveinfo-gw0_default Removing (cluster.py:147, run_and_check) 2026-04-10 19:23:50.261000 [ 639 ] DEBUG : Stderr: Volume roottestmasksensitiveinfo-gw0_data1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:23:50.261000 [ 639 ] DEBUG : Stderr: Network roottestmasksensitiveinfo-gw0_default Removed (cluster.py:147, run_and_check) 2026-04-10 19:23:50.261000 [ 639 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 19:23:50.288000 [ 639 ] DEBUG : Docker networks for project roottestmasksensitiveinfo-gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 19:23:50.317000 [ 639 ] DEBUG : Docker containers for project roottestmasksensitiveinfo-gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 19:23:50.345000 [ 639 ] DEBUG : Docker volumes for project roottestmasksensitiveinfo-gw0 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 19:23:50.345000 [ 639 ] DEBUG : Command:[docker container list --all --filter name='^/roottestmasksensitiveinfo-gw0-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 19:23:50.366000 [ 639 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 19:23:50.366000 [ 639 ] DEBUG : No running containers for project: roottestmasksensitiveinfo-gw0 (cluster.py:889, cleanup) 2026-04-10 19:23:50.366000 [ 639 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 19:23:50.392000 [ 639 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 19:23:50.393000 [ 639 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 19:23:50.429000 [ 639 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 19:23:50.429000 [ 639 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 19:23:50.429000 [ 639 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 19:23:50.429000 [ 639 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 19:23:50.457000 [ 639 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 19:23:50.457000 [ 639 ] DEBUG : Volumes pruned: 1 (cluster.py:925, cleanup) _____________________________ test_delete_cluster ______________________________ [gw8] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_delete_cluster(started_cluster): send_repeated_query("distributed") send_repeated_query("distributed2") > assert get_errors_count("test_cluster") > 0 E AssertionError: assert 0 > 0 E + where 0 = get_errors_count('test_cluster') test_reload_clusters_config/test.py:217: AssertionError ------------------------------ Captured log call ------------------------------- 2026-04-10 19:23:27.648000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:23:39.748000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:23:51.857000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:24:04.773000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:24:17.739000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:24:29.853000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:24:42.817000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:24:54.929000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:25:07.842000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:25:19.958000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:25:32.069000 [ 709 ] DEBUG : Executing query SELECT errors_count FROM system.clusters WHERE cluster='test_cluster' and host_name='node_1' on node (cluster.py:3658, query) ___________________________ test_update_one_cluster ____________________________ [gw8] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = def test_update_one_cluster(started_cluster): send_repeated_query("distributed") send_repeated_query("distributed2") > assert get_errors_count("test_cluster") > 0 E AssertionError: assert 0 > 0 E + where 0 = get_errors_count('test_cluster') test_reload_clusters_config/test.py:202: AssertionError ------------------------------ Captured log call ------------------------------- 2026-04-10 19:26:35.416000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:26:49.085000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:27:02.101000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:27:14.211000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:27:28.233000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:27:41.247000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:27:55.265000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:28:08.234000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:28:20.347000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:28:32.462000 [ 709 ] DEBUG : Executing query SELECT count() FROM distributed2 SETTINGS receive_timeout=1, handshake_timeout_ms=1 on node (cluster.py:3741, query_and_get_error) 2026-04-10 19:28:45.383000 [ 709 ] DEBUG : Executing query SELECT errors_count FROM system.clusters WHERE cluster='test_cluster' and host_name='node_1' on node (cluster.py:3658, query) ---------------------------- Captured log teardown ----------------------------- 2026-04-10 19:28:45.455000 [ 709 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/.env --project-name roottestreloadclustersconfig-gw8 --file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop --timeout 20] (cluster.py:121, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:28:53.096000 [ 709 ] DEBUG : Command:[bash -c [ -f /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] (cluster.py:121, run_and_check) 2026-04-10 19:28:53.111000 [ 709 ] DEBUG : Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/.env --project-name roottestreloadclustersconfig-gw8 --file /ClickHouse/tests/integration/test_reload_clusters_config/_instances-0-gw8/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml down --volumes] (cluster.py:121, run_and_check) 2026-04-10 19:28:53.611000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:28:53.612000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:28:53.612000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:28:53.612000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-node-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:28:53.612000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:28:53.612000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:28:53.612000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Stopping (cluster.py:147, run_and_check) 2026-04-10 19:28:53.612000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Stopped (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Removing (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo3-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo1-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Container roottestreloadclustersconfig-gw8-zoo2-1 Removed (cluster.py:147, run_and_check) 2026-04-10 19:28:53.613000 [ 709 ] DEBUG : Stderr: Network roottestreloadclustersconfig-gw8_default Removing (cluster.py:147, run_and_check) 2026-04-10 19:28:53.614000 [ 709 ] DEBUG : Stderr: Network roottestreloadclustersconfig-gw8_default Removed (cluster.py:147, run_and_check) 2026-04-10 19:28:53.614000 [ 709 ] DEBUG : Cleanup called (cluster.py:861, cleanup) 2026-04-10 19:28:53.636000 [ 709 ] DEBUG : Docker networks for project roottestreloadclustersconfig-gw8 are NETWORK ID NAME DRIVER SCOPE (cluster.py:840, print_all_docker_pieces) 2026-04-10 19:28:53.666000 [ 709 ] DEBUG : Docker containers for project roottestreloadclustersconfig-gw8 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:848, print_all_docker_pieces) 2026-04-10 19:28:53.688000 [ 709 ] DEBUG : Docker volumes for project roottestreloadclustersconfig-gw8 are DRIVER VOLUME NAME (cluster.py:856, print_all_docker_pieces) 2026-04-10 19:28:53.688000 [ 709 ] DEBUG : Command:[docker container list --all --filter name='^/roottestreloadclustersconfig-gw8-.*-1$' --format '{{.ID}}:{{.Names}}'] (cluster.py:121, run_and_check) 2026-04-10 19:28:53.710000 [ 709 ] DEBUG : Unstopped containers: {} (cluster.py:875, cleanup) 2026-04-10 19:28:53.710000 [ 709 ] DEBUG : No running containers for project: roottestreloadclustersconfig-gw8 (cluster.py:889, cleanup) 2026-04-10 19:28:53.710000 [ 709 ] DEBUG : Trying to prune unused networks... (cluster.py:895, cleanup) 2026-04-10 19:28:53.737000 [ 709 ] DEBUG : Trying to prune unused images... (cluster.py:911, cleanup) 2026-04-10 19:28:53.737000 [ 709 ] DEBUG : Command:[docker image prune -f] (cluster.py:121, run_and_check) 2026-04-10 19:28:53.777000 [ 709 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:145, run_and_check) 2026-04-10 19:28:53.777000 [ 709 ] DEBUG : Images pruned (cluster.py:914, cleanup) 2026-04-10 19:28:53.778000 [ 709 ] DEBUG : Trying to prune unused volumes... (cluster.py:920, cleanup) 2026-04-10 19:28:53.778000 [ 709 ] DEBUG : Command:[docker volume ls | wc -l] (cluster.py:121, run_and_check) 2026-04-10 19:28:53.807000 [ 709 ] DEBUG : Stdout:1 (cluster.py:145, run_and_check) 2026-04-10 19:28:53.808000 [ 709 ] DEBUG : Volumes pruned: 1 (cluster.py:925, cleanup) ----------------- generated report log file: parallel1_0.jsonl ----------------- ============================== slowest durations =============================== 308.13s call test_keeper_memory_soft_limit/test.py::test_soft_limit_create 137.03s call test_reload_clusters_config/test.py::test_add_cluster 130.03s call test_reload_clusters_config/test.py::test_update_one_cluster 124.54s call test_reload_clusters_config/test.py::test_delete_cluster 76.43s call test_keeper_mntr_pressure/test.py::test_aggressive_mntr 69.57s setup test_mask_sensitive_info/test.py::test_backup_table 63.12s call test_reload_clusters_config/test.py::test_simple_reload 58.50s call test_keeper_internal_secure/test.py::test_secure_raft_works_with_password 55.16s call test_mask_sensitive_info/test.py::test_backup_to_s3 34.04s call test_keeper_nodes_move/test.py::test_node_move 33.83s call test_keeper_block_acl/test.py::test_keeper_block_acl 28.15s setup test_reload_clusters_config/test.py::test_add_cluster 25.18s call test_jbod_ha/test.py::test_jbod_ha 24.32s teardown test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback 23.07s setup test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] 23.03s teardown test_keeper_disks/test.py::test_snapshots_with_disks 22.10s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] 21.84s call test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] 20.66s call test_executable_user_defined_functions_config_reload/test.py::test 20.65s call test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges 20.36s call test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] 19.91s teardown test_quorum_inserts/test.py::test_simple_add_replica 19.77s call test_keeper_disks/test.py::test_snapshots_with_disks 18.98s call test_keeper_disks/test.py::test_logs_with_disks 18.45s setup test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 18.08s call test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 17.73s setup test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers 17.40s call test_keeper_zookeeper_converter/test.py::test_acls[True] 17.23s call test_keeper_zookeeper_converter/test.py::test_smoke[True] 16.83s setup test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback 16.67s call test_quorum_inserts/test.py::test_insert_quorum_with_ttl 16.61s setup test_external_cluster/test.py::test_ddl 16.54s setup test_keeper_block_acl/test.py::test_keeper_block_acl 16.22s call test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] 15.97s call test_mask_sensitive_info/test.py::test_table_function_ways_to_call 15.88s call test_mask_sensitive_info/test.py::test_iceberg_cluster_function 15.70s teardown test_mask_sensitive_info/test.py::test_table_functions 15.44s teardown test_keeper_memory_soft_limit/test.py::test_soft_limit_create 15.01s call test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] 14.75s call test_filesystem_cache/test.py::test_custom_cached_disk 14.70s call test_keeper_internal_secure/test.py::test_secure_raft_works 14.70s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] 14.26s setup test_keeper_session/test.py::test_session_close_shutdown 13.86s call test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] 13.84s setup test_keeper_zookeeper_converter/test.py::test_acls[False] 13.80s setup test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 13.70s setup test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] 13.59s setup test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard 13.14s setup test_keeper_compression/test_with_compression.py::test_select 13.09s call test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task 12.92s call test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] 12.89s setup test_http_native/test.py::test_http_native_returns_timezone 12.79s teardown test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 12.64s setup test_external_http_authenticator/test.py::test_basic_auth_failed 12.57s call test_keeper_session/test.py::test_session_close_shutdown 12.43s setup test_jbod_ha/test.py::test_jbod_ha 12.42s setup test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task 12.09s call test_keeper_zookeeper_converter/test.py::test_smoke[False] 12.07s setup test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used 12.03s setup test_explain_estimates/test.py::test_explain_estimates 11.98s setup test_inherit_multiple_profiles/test.py::test_combined_profile 11.90s setup test_executable_user_defined_functions_config_reload/test.py::test 11.55s call test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers 11.52s setup test_filesystem_cache/test.py::test_custom_cached_disk 10.99s call test_keeper_zookeeper_converter/test.py::test_acls[False] 10.87s setup test_keeper_memory_soft_limit/test.py::test_soft_limit_create 10.52s setup test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success 9.62s setup test_keeper_four_word_command/test_allow_list.py::test_allow_list 9.55s call test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table 9.27s call test_keeper_persistent_log/test.py::test_state_duplicate_restart 8.95s setup test_keeper_disks/test.py::test_logs_with_disks 8.75s setup test_keeper_nodes_move/test.py::test_node_move 8.74s setup test_keeper_persistent_log/test.py::test_ephemeral_after_restart 8.35s teardown test_reload_clusters_config/test.py::test_update_one_cluster 8.33s setup test_grpc_protocol_ssl/test.py::test_insecure_channel 8.28s setup test_keeper_internal_secure/test.py::test_secure_raft_works 7.73s setup test_hot_reload_storage_policy/test.py::test_hot_reload_policy 7.60s setup test_keeper_mntr_pressure/test.py::test_aggressive_mntr 7.32s call test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table 6.51s call test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 6.47s teardown test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers 6.17s call test_filesystem_cache/test.py::test_keep_up_size_ratio 5.99s setup test_http_and_readonly/test.py::test_http_get_is_readonly 5.80s teardown test_external_cluster/test.py::test_ddl_replicated 5.47s teardown test_filesystem_cache/test.py::test_system_sync_filesystem_cache 5.03s call test_keeper_persistent_log/test.py::test_ephemeral_after_restart 4.95s call test_keeper_persistent_log/test.py::test_state_after_restart 4.58s call test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success 4.48s teardown test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass 4.31s teardown test_http_native/test.py::test_http_native_returns_timezone 4.25s teardown test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin 4.10s teardown test_explain_estimates/test.py::test_explain_estimates 3.97s teardown test_grpc_protocol_ssl/test.py::test_wrong_client_certificate 3.41s teardown test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table 3.22s teardown test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout 3.22s call test_mask_sensitive_info/test.py::test_encryption_functions 3.10s teardown test_http_and_readonly/test.py::test_http_get_is_readonly 3.02s call test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback 2.94s teardown test_inherit_multiple_profiles/test.py::test_combined_profile 2.93s call test_mask_sensitive_info/test.py::test_table_functions 2.90s call test_filesystem_cache/test.py::test_dynamic_resize 2.59s call test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] 2.54s teardown test_jbod_ha/test.py::test_jbod_ha 2.52s call test_filesystem_cache/test.py::test_system_sync_filesystem_cache 2.36s call test_hot_reload_storage_policy/test.py::test_hot_reload_policy 2.27s teardown test_keeper_mntr_pressure/test.py::test_aggressive_mntr 2.24s teardown test_keeper_block_acl/test.py::test_keeper_block_acl 2.21s call test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 2.21s call test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 2.12s call test_quorum_inserts/test.py::test_simple_add_replica 2.08s teardown test_keeper_nodes_move/test.py::test_node_move 2.08s call test_external_cluster/test.py::test_ddl 2.07s call test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout 2.05s call test_mask_sensitive_info/test.py::test_create_alter_user 2.01s call test_grpc_protocol_ssl/test.py::test_insecure_channel 2.00s call test_grpc_protocol_ssl/test.py::test_wrong_client_certificate 1.95s teardown test_keeper_internal_secure/test.py::test_secure_raft_works_with_password 1.92s teardown test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table 1.89s teardown test_keeper_session/test.py::test_session_timeout 1.88s teardown test_keeper_four_word_command/test_allow_list.py::test_allow_list 1.87s call test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 1.75s call test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 1.66s call test_external_http_authenticator/test.py::test_session_settings_from_auth_response 1.66s call test_keeper_four_word_command/test_allow_list.py::test_allow_list 1.62s call test_mask_sensitive_info/test.py::test_create_table 1.59s call test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 1.58s call test_mask_sensitive_info/test.py::test_create_database 1.55s call test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout 1.47s teardown test_filesystem_cache/test.py::test_custom_cached_disk 1.45s teardown test_keeper_persistent_log/test.py::test_state_duplicate_restart 1.43s teardown test_keeper_compression/test_with_compression.py::test_select 1.40s teardown test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success 1.40s teardown test_executable_user_defined_functions_config_reload/test.py::test 1.39s teardown test_keeper_zookeeper_converter/test.py::test_smoke[True] 1.27s call test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] 1.22s call test_mask_sensitive_info/test.py::test_create_dictionary 1.20s call test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used 1.19s call test_mask_sensitive_info/test.py::test_on_cluster 1.16s call test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] 1.14s call test_explain_estimates/test.py::test_explain_estimates 1.10s call test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 1.06s call test_insert_into_distributed_sync_async/test.py::test_insertion_sync 1.02s call test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout 0.99s call test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] 0.93s call test_filesystem_cache/test.py::test_filesystem_cache_log 0.92s call test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes 0.88s call test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage 0.79s call test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard 0.75s call test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk 0.68s call test_mask_sensitive_info/test.py::test_backup_table 0.63s call test_inherit_multiple_profiles/test.py::test_combined_profile 0.62s call test_external_http_authenticator/test.py::test_basic_auth_failed 0.50s call test_external_cluster/test.py::test_ddl_replicated 0.41s call test_external_http_authenticator/test.py::test_user_create_basic_auth_pass 0.40s call test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin 0.35s call test_keeper_session/test.py::test_session_timeout 0.24s call test_grpc_protocol_ssl/test.py::test_secure_channel 0.22s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] 0.19s call test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass 0.12s call test_keeper_compression/test_with_compression.py::test_select 0.12s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] 0.12s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] 0.12s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] 0.12s call test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] 0.11s call test_http_native/test.py::test_http_native_returns_timezone 0.11s call test_http_and_readonly/test.py::test_http_get_is_readonly 0.08s call test_external_http_authenticator/test.py::test_header_failed 0.00s teardown test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] 0.00s teardown test_mask_sensitive_info/test.py::test_backup_table 0.00s teardown test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] 0.00s setup test_reload_clusters_config/test.py::test_delete_cluster 0.00s teardown test_keeper_persistent_log/test.py::test_state_after_restart 0.00s teardown test_keeper_zookeeper_converter/test.py::test_acls[True] 0.00s teardown test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] 0.00s setup test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage 0.00s teardown test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] 0.00s teardown test_keeper_disks/test.py::test_logs_with_disks 0.00s teardown test_keeper_persistent_log/test.py::test_ephemeral_after_restart 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] 0.00s setup test_keeper_session/test.py::test_session_timeout 0.00s teardown test_keeper_session/test.py::test_session_close_shutdown 0.00s teardown test_keeper_zookeeper_converter/test.py::test_smoke[False] 0.00s teardown test_keeper_zookeeper_converter/test.py::test_acls[False] 0.00s setup test_reload_clusters_config/test.py::test_update_one_cluster 0.00s setup test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] 0.00s setup test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout 0.00s setup test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] 0.00s setup test_mask_sensitive_info/test.py::test_create_dictionary 0.00s teardown test_keeper_internal_secure/test.py::test_secure_raft_works 0.00s setup test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] 0.00s setup test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] 0.00s teardown test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] 0.00s teardown test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout 0.00s teardown test_reload_clusters_config/test.py::test_add_cluster 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 0.00s setup test_keeper_zookeeper_converter/test.py::test_smoke[True] 0.00s setup test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] 0.00s teardown test_grpc_protocol_ssl/test.py::test_secure_channel 0.00s teardown test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 0.00s setup test_filesystem_cache/test.py::test_keep_up_size_ratio 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] 0.00s setup test_quorum_inserts/test.py::test_simple_add_replica 0.00s teardown test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 0.00s teardown test_external_cluster/test.py::test_ddl 0.00s setup test_mask_sensitive_info/test.py::test_iceberg_cluster_function 0.00s setup test_grpc_protocol_ssl/test.py::test_wrong_client_certificate 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 0.00s teardown test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task 0.00s setup test_keeper_zookeeper_converter/test.py::test_smoke[False] 0.00s teardown test_mask_sensitive_info/test.py::test_create_database 0.00s setup test_keeper_zookeeper_converter/test.py::test_acls[True] 0.00s setup test_external_http_authenticator/test.py::test_header_failed 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 0.00s teardown test_reload_clusters_config/test.py::test_simple_reload 0.00s teardown test_mask_sensitive_info/test.py::test_iceberg_cluster_function 0.00s setup test_external_http_authenticator/test.py::test_session_settings_from_auth_response 0.00s setup test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] 0.00s setup test_keeper_internal_secure/test.py::test_secure_raft_works_with_password 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] 0.00s teardown test_external_http_authenticator/test.py::test_basic_auth_failed 0.00s setup test_filesystem_cache/test.py::test_filesystem_cache_log 0.00s setup test_keeper_persistent_log/test.py::test_state_duplicate_restart 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_ttl 0.00s setup test_keeper_disks/test.py::test_snapshots_with_disks 0.00s teardown test_mask_sensitive_info/test.py::test_table_function_ways_to_call 0.00s setup test_filesystem_cache/test.py::test_dynamic_resize 0.00s teardown test_mask_sensitive_info/test.py::test_encryption_functions 0.00s setup test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout 0.00s setup test_mask_sensitive_info/test.py::test_on_cluster 0.00s setup test_mask_sensitive_info/test.py::test_table_functions 0.00s setup test_keeper_persistent_log/test.py::test_state_after_restart 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] 0.00s setup test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table 0.00s teardown test_mask_sensitive_info/test.py::test_create_table 0.00s setup test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] 0.00s setup test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 0.00s setup test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_ttl 0.00s setup test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes 0.00s teardown test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard 0.00s setup test_external_cluster/test.py::test_ddl_replicated 0.00s teardown test_hot_reload_storage_policy/test.py::test_hot_reload_policy 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 0.00s teardown test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout 0.00s teardown test_reload_clusters_config/test.py::test_delete_cluster 0.00s setup test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] 0.00s setup test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin 0.00s teardown test_filesystem_cache/test.py::test_dynamic_resize 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] 0.00s setup test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges 0.00s teardown test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used 0.00s teardown test_mask_sensitive_info/test.py::test_create_alter_user 0.00s setup test_mask_sensitive_info/test.py::test_create_database 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] 0.00s setup test_mask_sensitive_info/test.py::test_table_function_ways_to_call 0.00s teardown test_mask_sensitive_info/test.py::test_on_cluster 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 0.00s setup test_filesystem_cache/test.py::test_system_sync_filesystem_cache 0.00s teardown test_filesystem_cache/test.py::test_keep_up_size_ratio 0.00s setup test_external_http_authenticator/test.py::test_user_create_basic_auth_pass 0.00s setup test_grpc_protocol_ssl/test.py::test_secure_channel 0.00s setup test_mask_sensitive_info/test.py::test_create_alter_user 0.00s teardown test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk 0.00s teardown test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] 0.00s teardown test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] 0.00s setup test_mask_sensitive_info/test.py::test_backup_to_s3 0.00s setup test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout 0.00s setup test_mask_sensitive_info/test.py::test_create_table 0.00s teardown test_grpc_protocol_ssl/test.py::test_insecure_channel 0.00s teardown test_insert_into_distributed_sync_async/test.py::test_insertion_sync 0.00s setup test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass 0.00s setup test_mask_sensitive_info/test.py::test_encryption_functions 0.00s setup test_insert_into_distributed_sync_async/test.py::test_insertion_sync 0.00s setup test_reload_clusters_config/test.py::test_simple_reload 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 0.00s teardown test_mask_sensitive_info/test.py::test_create_dictionary 0.00s teardown test_external_http_authenticator/test.py::test_session_settings_from_auth_response 0.00s call test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] 0.00s teardown test_mask_sensitive_info/test.py::test_backup_to_s3 0.00s teardown test_filesystem_cache/test.py::test_filesystem_cache_log 0.00s call test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] 0.00s teardown test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage 0.00s teardown test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] 0.00s teardown test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] 0.00s teardown test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes 0.00s teardown test_external_http_authenticator/test.py::test_header_failed 0.00s teardown test_external_http_authenticator/test.py::test_user_create_basic_auth_pass =========================== short test summary info ============================ FAILED test_mask_sensitive_info/test.py::test_create_table - assert 'DNS_ERRO... FAILED test_mask_sensitive_info/test.py::test_table_functions - helpers.clien... FAILED test_reload_clusters_config/test.py::test_delete_cluster - AssertionEr... FAILED test_reload_clusters_config/test.py::test_update_one_cluster - Asserti... PASSED test_external_http_authenticator/test.py::test_basic_auth_failed PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used PASSED test_external_http_authenticator/test.py::test_header_failed PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes PASSED test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard PASSED test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk PASSED test_external_http_authenticator/test.py::test_session_settings_from_auth_response PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2] PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1] PASSED test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin PASSED test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2] PASSED test_external_http_authenticator/test.py::test_user_create_basic_auth_pass PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync PASSED test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass PASSED test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path] PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout PASSED test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout PASSED test_keeper_zookeeper_converter/test.py::test_acls[False] PASSED test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] PASSED test_grpc_protocol_ssl/test.py::test_insecure_channel PASSED test_grpc_protocol_ssl/test.py::test_secure_channel PASSED test_grpc_protocol_ssl/test.py::test_wrong_client_certificate PASSED test_keeper_persistent_log/test.py::test_ephemeral_after_restart PASSED test_external_cluster/test.py::test_ddl PASSED test_external_cluster/test.py::test_ddl_replicated PASSED test_keeper_persistent_log/test.py::test_state_after_restart PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0] PASSED test_filesystem_cache/test.py::test_custom_cached_disk PASSED test_keeper_zookeeper_converter/test.py::test_acls[True] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1] PASSED test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2] PASSED test_filesystem_cache/test.py::test_dynamic_resize PASSED test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection PASSED test_filesystem_cache/test.py::test_filesystem_cache_log PASSED test_keeper_persistent_log/test.py::test_state_duplicate_restart PASSED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] PASSED test_hot_reload_storage_policy/test.py::test_hot_reload_policy PASSED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] PASSED test_keeper_disks/test.py::test_logs_with_disks PASSED test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False] PASSED test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table PASSED test_keeper_session/test.py::test_session_close_shutdown PASSED test_keeper_session/test.py::test_session_timeout PASSED test_quorum_inserts/test.py::test_insert_quorum_with_ttl PASSED test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges PASSED test_quorum_inserts/test.py::test_simple_add_replica PASSED test_mask_sensitive_info/test.py::test_backup_table PASSED test_keeper_disks/test.py::test_snapshots_with_disks PASSED test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage PASSED test_explain_estimates/test.py::test_explain_estimates PASSED test_filesystem_cache/test.py::test_keep_up_size_ratio PASSED test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task PASSED test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True] PASSED test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts PASSED test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table PASSED test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node] PASSED test_inherit_multiple_profiles/test.py::test_combined_profile PASSED test_filesystem_cache/test.py::test_system_sync_filesystem_cache PASSED test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False] PASSED test_executable_user_defined_functions_config_reload/test.py::test PASSED test_http_and_readonly/test.py::test_http_get_is_readonly PASSED test_http_native/test.py::test_http_native_returns_timezone PASSED test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success PASSED test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True] PASSED test_keeper_four_word_command/test_allow_list.py::test_allow_list PASSED test_keeper_internal_secure/test.py::test_secure_raft_works PASSED test_keeper_zookeeper_converter/test.py::test_smoke[False] PASSED test_mask_sensitive_info/test.py::test_backup_to_s3 PASSED test_mask_sensitive_info/test.py::test_create_alter_user PASSED test_mask_sensitive_info/test.py::test_create_database PASSED test_jbod_ha/test.py::test_jbod_ha PASSED test_mask_sensitive_info/test.py::test_create_dictionary PASSED test_mask_sensitive_info/test.py::test_encryption_functions PASSED test_keeper_zookeeper_converter/test.py::test_smoke[True] PASSED test_keeper_nodes_move/test.py::test_node_move PASSED test_keeper_block_acl/test.py::test_keeper_block_acl PASSED test_mask_sensitive_info/test.py::test_iceberg_cluster_function PASSED test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback PASSED test_mask_sensitive_info/test.py::test_on_cluster PASSED test_keeper_compression/test_with_compression.py::test_select PASSED test_reload_clusters_config/test.py::test_add_cluster PASSED test_mask_sensitive_info/test.py::test_table_function_ways_to_call PASSED test_keeper_internal_secure/test.py::test_secure_raft_works_with_password PASSED test_keeper_mntr_pressure/test.py::test_aggressive_mntr PASSED test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers PASSED test_reload_clusters_config/test.py::test_simple_reload PASSED test_keeper_memory_soft_limit/test.py::test_soft_limit_create =================== 4 failed, 96 passed in 493.85s (0:08:13) =================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 492, in subprocess.check_call(cmd, shell=True, bufsize=0) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_8t0hyd --privileged --dns-search='.' --memory=30709035008 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=5ccda723c1fc -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=d862517635bf -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 --report-log=parallel1_0.jsonl --report-log-exclude-logs-on-passed-tests 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_key[dod_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[cluster_node2]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node1]' 'test_distributed_over_distributed/test.py::TestDistributedOverDistributedSuite::test_select_with_order_by_node[dod_node2]' test_executable_user_defined_functions_config_reload/test.py::test test_explain_estimates/test.py::test_explain_estimates test_external_cluster/test.py::test_ddl test_external_cluster/test.py::test_ddl_replicated test_external_http_authenticator/test.py::test_basic_auth_failed test_external_http_authenticator/test.py::test_header_failed test_external_http_authenticator/test.py::test_session_settings_from_auth_response test_external_http_authenticator/test.py::test_user_create_basic_auth_pass test_external_http_authenticator/test.py::test_user_from_config_basic_auth_pass test_fetch_partition_with_outdated_parts/test.py::test_fetch_partition_with_outdated_parts 'test_filesystem_cache/test.py::test_caches_with_the_same_configuration[node]' 'test_filesystem_cache/test.py::test_caches_with_the_same_configuration_2[node_caches_with_same_path]' test_filesystem_cache/test.py::test_custom_cached_disk test_filesystem_cache/test.py::test_dynamic_resize test_filesystem_cache/test.py::test_filesystem_cache_log test_filesystem_cache/test.py::test_force_filesystem_cache_on_merges test_filesystem_cache/test.py::test_keep_up_size_ratio 'test_filesystem_cache/test.py::test_parallel_cache_loading_on_startup[node]' test_filesystem_cache/test.py::test_system_sync_filesystem_cache test_grpc_protocol_ssl/test.py::test_insecure_channel test_grpc_protocol_ssl/test.py::test_secure_channel test_grpc_protocol_ssl/test.py::test_wrong_client_certificate test_hot_reload_storage_policy/test.py::test_hot_reload_policy test_hot_reload_storage_policy/test.py::test_hot_reload_policy_distributed_table test_http_and_readonly/test.py::test_http_get_is_readonly test_http_native/test.py::test_http_native_returns_timezone test_inherit_multiple_profiles/test.py::test_combined_profile test_insert_distributed_async_extra_dirs/test.py::test_insert_distributed_async_send_success test_insert_into_distributed_sync_async/test.py::test_async_inserts_into_local_shard test_insert_into_distributed_sync_async/test.py::test_insertion_sync test_insert_into_distributed_sync_async/test.py::test_insertion_sync_fails_with_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_sync_with_disabled_timeout test_insert_into_distributed_sync_async/test.py::test_insertion_without_sync_ignores_timeout test_jbod_ha/test.py::test_jbod_ha test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_detect_background_changes test_jbod_load_balancing/test.py::test_jbod_load_balancing_least_used_next_disk test_jbod_load_balancing/test.py::test_jbod_load_balancing_round_robin test_keeper_block_acl/test.py::test_keeper_block_acl test_keeper_compression/test_with_compression.py::test_select test_keeper_disks/test.py::test_logs_with_disks test_keeper_disks/test.py::test_snapshots_with_disks test_keeper_four_word_command/test_allow_list.py::test_allow_list test_keeper_internal_secure/test.py::test_secure_raft_works test_keeper_internal_secure/test.py::test_secure_raft_works_with_password test_keeper_memory_soft_limit/test.py::test_soft_limit_create test_keeper_mntr_pressure/test.py::test_aggressive_mntr test_keeper_nodes_move/test.py::test_node_move test_keeper_persistent_log/test.py::test_ephemeral_after_restart test_keeper_persistent_log/test.py::test_state_after_restart test_keeper_persistent_log/test.py::test_state_duplicate_restart test_keeper_session/test.py::test_session_close_shutdown test_keeper_session/test.py::test_session_timeout 'test_keeper_zookeeper_converter/test.py::test_acls[False]' 'test_keeper_zookeeper_converter/test.py::test_acls[True]' 'test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[False]' 'test_keeper_zookeeper_converter/test.py::test_multi_and_failed_requests[True]' 'test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[False]' 'test_keeper_zookeeper_converter/test.py::test_simple_crud_requests[True]' 'test_keeper_zookeeper_converter/test.py::test_smoke[False]' 'test_keeper_zookeeper_converter/test.py::test_smoke[True]' test_mask_sensitive_info/test.py::test_backup_table test_mask_sensitive_info/test.py::test_backup_table_AzureBlobStorage test_mask_sensitive_info/test.py::test_backup_to_s3 test_mask_sensitive_info/test.py::test_create_alter_user test_mask_sensitive_info/test.py::test_create_database test_mask_sensitive_info/test.py::test_create_dictionary test_mask_sensitive_info/test.py::test_create_table test_mask_sensitive_info/test.py::test_encryption_functions test_mask_sensitive_info/test.py::test_iceberg_cluster_function test_mask_sensitive_info/test.py::test_on_cluster test_mask_sensitive_info/test.py::test_table_function_ways_to_call test_mask_sensitive_info/test.py::test_table_functions test_mutation_fetch_fallback/test.py::test_mutation_fetch_fallback test_mutations_with_merge_tree/test.py::test_mutations_with_merge_background_task test_mutations_with_merge_tree/test.py::test_mutations_with_truncate_table test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica test_reload_auxiliary_zookeepers/test.py::test_reload_auxiliary_zookeepers test_reload_clusters_config/test.py::test_add_cluster test_reload_clusters_config/test.py::test_delete_cluster test_reload_clusters_config/test.py::test_simple_reload test_reload_clusters_config/test.py::test_update_one_cluster 'test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_log_table[s3_plain]' test_remote_blobs_naming/test_backward_compatibility.py::test_read_new_format 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case0]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case1]' 'test_remote_blobs_naming/test_backward_compatibility.py::test_replicated_merge_tree[test_case2]' -vvv " altinityinfra/integration-tests-runner:226bfaf75ac1 ' returned non-zero exit status 1.